M Mosaic-based Relevance Feedback for Image Retrieval

A standard approach for content-based image retrieval (CBIR) is based on the extraction and comparison of features usually related to dominant colours, shapes, textures and layout (Del Bimbo, 1999). These features are a-priori defined and extracted, when the image is inserted into the database. At query time the user submits a similar sample image (query-by-sample-image) or draws a sketch (query-by-sketch) of the sought archived image. The similarity degree of the current query image and the target images is determined by calculation of a multidimensional distance between the corresponding features. The computed similarity values allow the creation of an image ranking, where the first k, usually k=32 or k=64, images are considered retrieval hits. These are chained in a list called ranking and then presented to the user. Each of these images can be used as a starting point for a refined search in order to improve the obtained results. The assessment of the retrieval result is based on a subjective evaluation of whole images and their position in the ranking. An important disadvantage of the retrieval with content-based features and the presentation of the resulting images as ranking is that the user is usually not aware, why certain images are shown on the top positions and why certain images are ranked low or not presented at all. Furthermore, users are also interested which sketch properties are decisive for the consideration and rejection of the images, respectively. In case of primitive features like colour these questions can be often answered intuitively. Retrieval with complex features considering for example texture and layout creates rankings, where the similarity between the query and the target images is not always obvious. Thus, the user is not satisfied with the displayed results and would like to improve the query, but it is not clear to him/her, which parts of the querying sketch or the sample image should be modified and improved according to the desired targets. Therefore, a suitable feedback mechanism is necessary. BACKGROUND